Digital watermarking is a process for modifying media content to embed a machine-readable code into the data content. The data may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media such as images, audio signals, and video signals. However, it may also be applied to other types of data, including documents (e.g., through line, word or character shifting), software, multi-dimensional graphics models, and surface textures of objects.
Digital watermarking systems have two primary components: an embedding component that embeds the watermark in the media content, and a reading component that detects and reads the embedded watermark. The embedding component embeds a watermark pattern by altering data samples of the media content in the spatial or frequency domains. The reading component analyzes target content to detect whether a watermark pattern is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.
Recently, digital watermarks have been used in applications for encoding auxiliary data in video, audio and still images. Despite the pervasiveness of multimedia content, such applications generally focus on ways to embed and detect watermarks in a single media type.
As an auxiliary data channel within multimedia content, digital watermarks may be used for carrying various types of data, including content identifiers as well as other types of data. A complementary type of content identification is a content hash, which is derived from attributes of a content signal, and when used in conjunction with a database of hashes, can be used to identify and associate metadata with the content signal.
This disclosure describes various applications of decoding auxiliary data in multimedia content with two or more media signals of different media types. These applications include using watermarks decoded from different media type signals within multimedia content to control processing of the multimedia content. One application is to use the watermark in one media signal to locate the watermark in another media signal. This is applicable to movies where a watermark in one media signal, such as the audio or video track, is used to locate the watermark in another media signal.
The watermark messages from different media signals may be combined for a variety of applications. One such application is to control processing of the multimedia signal. For example, the combined message can be used to control playback, copying or recording of the multimedia content.
Another application is copy control of multimedia content where a watermark from one media signal is used to control processing of the multimedia content. An audio watermark may be used to control processing of the video signal in a movie, or a video watermark may be used to control processing of the audio signal in the movie.
Another variation is where a watermark decoded from a first media signal of a first media type is used to decode a second media signal. The first and second media signals may be of the same or different types. Also, they may be part of the same composite media signal, such as an audio or video sequence. The term, “composite,” refers to a collection of media signals, which may be temporal portions (e.g., time frames in audio or video), or spatial portions (e.g., blocks of pixels in an image or video frame) of a visual, audio, or audio visual work. As an example, the first media signal may be an audio or video frame (or frames) in an audio or video sequence and the second media signal may be subsequent frames in the same sequence.
This approach is used in a variety of applications, such as content de-scrambling, decryption, and decompression. For example, a watermark in the first media signal is used to de-scramble, decrypt, or decompress the second media signal. In addition, the watermark in the first media signal may be used to decode a different watermark from the second signal.
Another method uses a watermark decoded from a first media signal of a first media type to decode metadata associated with the first media signal. In one case, the watermark is used to locate the metadata, which may be hidden for security purposes, or distributed throughout the multimedia content at locations (e.g., video frames or scenes, or at audio clips) that the metadata pertains to. The metadata located from the watermark may be located on the same storage medium that includes the first media signal. For example, the metadata may be located on portable storage device, such as flash memory, a magnetic memory device (e.g., tape or disk), or an optical memory device (e.g., CD, DVD, minidisk, etc.). The metadata may be located in a file header or some other place (e.g., encoded in the disk wobble).
There are a variety of applications of the watermark in this context. It may carry a key to decrypt, decompress, descramble, or locate the metadata. The metadata, in turn, may be used to control processing of the media signal in a computer or consumer electronic device. For example, it may be used to control usage rights, playback, recording, copying, transfer, etc.
Yet another method decodes first and second watermarks and forms a key for decoding data from the first and second watermarks. The watermarks may be decoded from the same or different media signals. For example, the watermarks may be decoded from media signals from the same composite signal. They may be derived from different types of media signals, such as the audio and video tracks of a movie. Alternatively, they may be derived from different parts of the same type of media signal, such as an audio sequence, video sequence, or image. The watermarks may be extracted from a signal or signals stored in a storage device, such as a portable storage device (e.g., optical or magnetic disk or tape, flash memory, etc.).
The key formed from the watermarks may be used for a variety of applications. It may be used as a watermark key to decode a watermark from a media signal. It may be used as a decryption or de-scrambling key. Also, it may be used as a decompression key (e.g., a parameter used to decompress a media signal).
One aspect of the invention is a method for using a digital watermark encoded in a first media signal to synchronize play back of a different media signal. This method decodes auxiliary data from the first media signal, where the auxiliary data includes at least one digital watermark extracted from the first media signal. The method retrieves the different media signal and derives data for synchronizing playback of the first media signal and the different media signal from the digital watermark. The method controls playback of the different media signal together with the first media signal using the digital watermark to synchronize playback of the first and different media signals on an output device.
Another aspect of the invention is a method for using a digital watermark in a media track to synchronize play back of different media data from the track. This method decodes auxiliary data from the media track, where the auxiliary data includes text data and at least one digital watermark extracted from a media signal in the media track. It reads media data from the track and derives data for synchronizing playback of the media and text data from the digital watermark. The method controls playback of the media data using the digital watermark to synchronize playback of the text and the media data.
Further features of the invention will become apparent with reference to the following detailed description and accompanying drawings.